Robot Howitzer

Really Awesome
A Project By Xiang Lan and Ebrahim Shah.

Demonstration Video

Introduction

The Nerf Howitzer is a nerf air gun mounted on a white platform that sits on a robot chassis. This chassis has two wheels powered by DC motors. The nerf gun is operated using two standard servo motors: one to press the trigger and one to change the vertical angle of the gun i.e. elevation of the barrel. A picamera is used to take images as the program is executed and the robot chassis turns both left and right. The images are then processed using the opencv2 library for object detection and distance measurement. A voice recognition library called vosk is also used to convert sound into a string for aiming and firing. The nerf howitzer uses a recording as opposed to an actual voice command and turns towards the direction of the target. The program then waits for a voice input which was also replaced by a recording and prompts the servo to turn causing the gun to fire.

Project Objective

Build a machine that can use a camera to detect objects and find their distance.
Add mobility to change the robot’s position relative to the target to aim better .
Use a mechanism to launch a projectile upon detecting and aiming at a target .

Design

The nerf howitzer system can be divided into three blocks for a better understanding of the system:

Fig.1 Three Blocks of the whole system

A top-bottom view of some of the components. From left to right: piTFT mouned on a robot chassis, a packet of steel brackets for mounting the gun platform, two black parallax servo motors to control the gun angle and trigger, the NERF Rival gun.

Fig.2 components

The design can be understood by a chronological breakdown of the functions:

Sweep

The howitzer begins facing forward and performs a ‘sweep’ maneuver where it turns in a semicircle first to its left (position 1)i.e. upto 90 degrees and then back to the right (position 2) pausing at its original forward facing position before continuing to the right (position 3) and cycling back to its original forward facing position.

At each position the howitzer stops for 2 seconds and the front mounted picamera takes a photo which is saved.

Fig.3 A drawing of the sweep maneuver. At each cross, the picamera takes a photo

Target Detection

The howitzer verifies if an image has a target
If a target is present, the distance to that target is calculated

The first step is accomplished with the help of a function called ‘find_distance” that uses opencv to detect any black shapes and draw a contour around them all. It then assigns all black targets to an array which are returned. The ‘find marker’ function returns the size and position of the target.

The second step is achieved using a function called ‘find_distance’. This function uses a reference image where the distance is known, to find a calibration factor between an object’s size on the screen and its approximate distance. This is then compared to the output of ‘find_marker’ to derive the distance value.

Fig.4 The picamera mounted on a piece of foam board facing straight ahead and connected to the raspberry pi using the white ribbon cable

Fig.5 A processed image showing a red outline of the largest contour within the green dots which the program classifies as smaller objects. On the bottom right the calculated distance ‘0.05’ ft is shown.

Voice Recognition

Based on the microphone, the robot can listen to the voice and store it in voice_wav directory. Then using the wav format audio file we just saved as input of ‘get_results()’ to implement voice recognition. ‘get_results()’ is defined in voiceControl.py, and this function use vosk model as a tool to obtain text message from audio files. In ‘voiceControl()’ function, we processed the return value of ‘get_results()’ to get commands which can be understood by our robot and call ‘aim()’ or ‘fire()’ functions based on different commands. Based on results of voice recognition, we can do more things, including control robot to move and turn, etc.

Turning and Aiming

Once the target has been identified and the howitzer is at its rest position, it proceeds to turn in the direction of the target using the function ‘turn_to_target’ which takes in an integer i.e. 0 for left, 2 for right. There is no need for a third integer since the howitzer is already facing forward.

It then waits for a voice command ‘aim’ that is processed into a string using the function voiceControl.py which relies on the vosk library. This prompts the servo controlling the angle of the nerf gun to rotate.

By default, the gun is inclined at an angle. The ‘aiming’ function takes the distance as an input. It then rotates the servo to lower the gun based on how close the target is. For simplicity, there are three levels; lower all the way for less than 5 inches, lower half way for less than 10 inches, don’t lower for greater than 10 inches.

The foam core platform held together by duct tape. The white cutout acts like a hinge which is pulled back by the blue wire connected to a servo motor. This is connected to the rear handle of the nerf gun. The aiming function relaxes the wire raising the height of the rear handle and lowering the sights of the gun

Fig.6 foam core platform

Fire

Finally, the howitzer uses a voice input to activate the servo connected to the trigger. Using the voiceControl() function, the speech input of ‘fire’ gets converted to a string. An if condition in the fire() function, checks for the word before using PWM to turn the servo until the trigger is pressed and reset it.

The mounted gun is connected using a real metal hinge at the base of it’s front handle which acts as a pivot as the rear handle which rests on the white foam core cut out in the image above. The black motor on the rear handle pulls the trigger when the fire function is called.

Fig.7 foam core platform

Drawings

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum lorem nulla, consectetur at leo vel, pretium bibendum nisl. Cras blandit quam a enim ultrices, eu convallis enim posuere. Donec eleifend enim sed purus consectetur, vitae cursus lectus varius. Vivamus consectetur felis nec est venenatis posuere. Phasellus vitae aliquet erat. In laoreet lacinia mollis. Quisque iaculis nisl fermentum pharetra lobortis. Donec rhoncus dui sem, ac molestie leo tristique vel. Phasellus in nibh feugiat, fringilla lectus in, elementum magna. Etiam quis dui condimentum, tempus ex in, dapibus est. Cras ut congue augue. Donec ac enim ex. Ut id tristique risus, vel porttitor quam. Sed ultricies enim eu nibh porttitor, vel sodales enim feugiat. Fusce volutpat venenatis magna ac ultrices. Curabitur eget urna ut nulla mattis convallis non eu diam.

Testing

Video vs. Image processing

Initially, the plan was to stream live video and show targets on it. However, it proved to be more complicated than thought. To get around this problem, it was decided to take images and run target detection on the image rather than do it on a video.

Voice Recognition

At first, we tried to use Sphinx.google to implement voice recognition job, however, it works well only when it is online. In order to run this project at any time and have a better application scenario, we tried different methods to recognize audio files instead of Sphinx. With vosk model, input audio file, we can get result with a great accuracy rate which is totally enough for us to control the motors

Accuracy of nerf gun

Since the gun is too large and the mechanism to control it is very inadequate, accuracy was severely compromised. During testing, the most reasonable option was to use a folded foam-core cut out to change the vertical angle of the gun. The range of this angle is hardly 30 degrees which does little to control the accuracy of the gun.

Trigger pull force

The servo used to pull the trigger did not have enough torque to complete the action and often the gun didn’t fire. To resolve this, an elastic rubber band was cut and tied to the trigger which pulled it back to the edge so that a little torque would be needed to overpower the trigger spring.

Movement of wheels

Due to excess weight from the oversized gun on the small robot, it was easy to understand why the wheels did not move easily and this reduced the sweep action. At times, the robot failed to complete any turns and had to be lifted to confirm if the motors are turning. To resolve this, the duty cycle of the PWM channels was set to a maximum i.e. 100% and new AA batteries were used to eliminate any problems. Another option would have been to use more powerful motors rather than the basic DC motors which were inadequate for this application.

Result

The completed prototype is a robot chassis powered by two DC motors, that supports a nerf gun outfitted with two servo motors controlling the gun’s elevation and trigger. A picamera is used to sense targets on a white background and the raspberry pi processes those images to determine where those targets are located. The robot then uses a microphone for a voice input to ‘aim’ which prompts it to turn towards the target. It automatically sends a PWM signal to the servo to lower the eAt this point, it once again waits for a voice input of ‘fire’ before sending a PWM signal to the servo and pulling the trigger. The raspberry pi then controls the servos guiding the aim of the gun and finally firing it.

Right figment is the sideview of the nerf howitzer. The gun is at its default angle and pivots at the base of the magazine to lower the angle of its barrel.

Fig.8 Robot Howitzer

Work Distribution

Ebrahim implemented target detection and distance calculation, and xiang implemented voice recognition and control. We did programming and most of the core logic code together, including but not limited to robot movement and firing. That was an excellent experience with Ebrahim and Professor Skovira in ECE5725!

Xiang Lan

xl837@cornell.edu

Ebrahim Shah

ss3532@cornell.edu

References

PiCamera Document
Bootstrap
Pigpio Library
R-Pi GPIO Document
Sphinx Document
Vosk Document
About find_distance()